LOCAL GEOMETRIZED RANKIN-SELBERG METHOD FOR GL(n)

Following G. Laumon [12], to a nonramified `-adic local system E of rank n on a curve X one associates a complex of `-adic sheaves nKE on the moduli stack of rank n vector bundles on X with a section, which is cuspidal and satisfies the Hecke property for E. This is a geometric counterpart of the well-known construction due to J. Shalika [19] and I. Piatetski-Shapiro [18]. We express the cohomology of the tensor product nKE1⊗nKE2 in terms of cohomology of the symmetric powers of X. This may be considered as a geometric interpretation of the local part of the classical RankinSelberg method for GL(n) in the framework of the geometric Langlands program. 0. Introduction This is the first in a series of two papers, where we propose a geometric version of the classical Rankin-Selberg method for computation of the scalar product of two cuspidal automorphic forms on GL(n) over a function field. This geometrization fits in the framework of the geometric Langlands program initiated by V. Drinfeld, A. Beilinson, and Laumon. Let X be a smooth, projective, geometrically connected curve over Fq . Let ` be a prime invertible in Fq . According to the Langlands correspondence for GL(n) over function fields (proved by L. Lafforgue), to any smooth geometrically irreducible Q̄`sheaf E of rank n on X is associated a (unique up to a multiple) cuspidal automorphic form φE : Bunn(Fq) → Q̄`, which is a Hecke eigenvector with respect to E . The function φE is defined on the set Bunn(Fq) of isomorphism classes of rank n vector bundles on X . The classical method of Rankin and Selberg for GL(n) may be divided into two parts: local and global. The global result calculates for any integer d the scalar product DUKE MATHEMATICAL JOURNAL Vol. 111, No. 3, c © 2002 Received 9 June 2000. Revision received 21 February 2001. 2000 Mathematics Subject Classification. Primary 11R39; Secondary 11S37, 11F70, 14H60.